Papers with surrogate model
Transferable Direct Prompt Injection via Activation-Guided MCMC Sampling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show superior cross-model transferability . Prompt injection attacks are among the most critical threats . |
| Approach: | They propose an activations-guided prompt injection attack framework to address the impracticality of existing white-box/gray-box methods and the poor transferability of black-box approaches. |
| Outcome: | The proposed framework achieves 49.6% success rate and 34.6% improvement over human-crafted prompts on five mainstream LLMs. |
Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text embedding models have significantly streamlined the process of generating embeddables. |
| Approach: | They develop a transfer attack method that uses a surrogate model to mimic the victim model's behavior and infers sensitive information from embeddings without direct access. |
| Outcome: | The proposed method outperforms existing methods and reveals potential privacy vulnerabilities in embedding technologies. |
Crafting Adversarial Inputs for Large Vision-Language Models Using Black-Box Optimization (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing white-box jailbreak methods require full model accessibility and require computational costs. |
| Approach: | They propose a black-box jailbreak attack using Zeroth-Order optimization using ZO-SPSA. |
| Outcome: | The proposed method achieves highest jailbreak success rate on three LVLMs, including InstructBLIP, LLaVA and MiniGPT-4. |
Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are powerful zero-shot assessors used in real-world situations . however, no study has examined the vulnerability of judge-LLM to adversarial manipulation . |
| Approach: | They propose a simple surrogate attack where a surrogated model is attacked and the learned attack phrase transferred to unknown judge-LLMs. |
| Outcome: | The proposed algorithm shows that judge-LLMs can be significantly more susceptible to adversarial attacks when used for absolute scoring, rather than comparative assessment. |
Q-FAKER: Query-free Hard Black-box Attack via Controlled Generation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing adversarial approaches to verify the vulnerability of language models require numerous queries and the information on the target model. |
| Approach: | They propose a method that generates adversarial examples without accessing the target model and uses controlled generation techniques to generate adversarials. |
| Outcome: | The proposed method generates adversarial examples without accessing the target model and is highly transferable and high quality. |
Profiler: Black-box AI-generated Text Origin Detection via Context-aware Inference Pattern Analysis (2025.emnlp-main)
Copied to clipboard
Hanxi Guo, Siyuan Cheng, Xiaolong Jin, Zhuo Zhang, Guangyu Shen, Kaiyuan Zhang, Shengwei An, Guanhong Tao, Xiangyu Zhang
| Challenge: | Existing methods to identify the origin of AI-generated texts fail to identify origin due to the high similarity of different LLMs. |
| Approach: | They propose a black-box AI-generated text origin detection method which accurately predicts the origin of an input text by extracting distinct context inference patterns. |
| Outcome: | The proposed method outperforms 10 state-of-the-art baselines and achieves a 25% increase in AUC score on average across natural language and code datasets. |